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DETAILED ACTION 



1. 



This action is responsive to communications: Amendment filed 6/03/04. 



2. 



Claims 1-28 are pending. Claims 1,17, and 21 are independent claims. 



Claim Rejections - 35 USC §103 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-4, 8, 12-13, 16-21, 23-25, and 27-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Mukhopadhyay et al. ("Passive Capture and Structuring of 
Lectures", 10/1999, ACM Multimedia '99, pages 1-11 (submitted in the IDS filed on 
9/14/00)) in view of Mitchell et al. (USPN 5,963,966 - issued on 10/1999). 

Regarding independent claim 1 and (dependent claims 13, 23, 25, and 27), Mukhopadhyay 
discloses: 

A method for linking a document to a video (Mukhopadhyay on page 2, right column teaches 
slides are synchronized with the video; on page 3, left column teaches slides are matched to the 
video; and on page 10, left column teaches inserting digitized output of scan converters when 
demonstrations are given), comprising the steps of: 

document identifier (Mukhopadhyay on page 5, see figure 7 teaches each slide image are 
set with a value to match with each video frame); 
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obtaining a video file having a plurality of video frames (Mukhopadhyay on page 5 
teaches video sequence of a plurality of segments); 

transforming the plurality of video frames into plurality of respective video frame 
identifiers (Mukhopadhyay on page 5 teaches matching a representative frame from each 
segment in T to one of the slides in S; a video sequence comprises a set of time values T ); 

comparing the document identifier with the plurality of video frame identifiers 
(Mukhopadhyay on page 5 teaches computing a video sequence of v(t) segments or frames to 
match against every slide image S (see figure 7)); and 

linking the document to a first video frame in the plurality of video frames 
(Mukhopadhyay on page 5, see figure 7 teaches matching slides in each frame to synchronize 
video data to the slides). 

However, Mukhopadhyay does not explicitly disclose "obtaining a scanned document 
having margins", "removing margins from scanned documents", "scaling the scanned document" 
and "transforming the scanned document". 

Mitchell discloses "obtaining a scanned document having margins", "removing margins 
from scanned documents" and "scaling the scanned document" on col. 8, lines 36-43 and col 9, 
lines 18-19 teaches document pages are scanned into images; wherein the images are cropped to 
remove the borders and rescaled; "transforming the scanned document" on col. 6, lines 36-40 
teaches translating from paper to electronic form such as SGML, HTML, and text. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 



1 
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Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 2, Mitchell discloses scanning a document to create an image on 
col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 3, Mitchell discloses cropping the image from the scanned 
document to remove the borders on col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 4, Mukhopadhyay discloses a reduced image of the slide displayed 
on the Lecture Browser user interface to fit the corresponding video within the user interface (see 
figure 2), 
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Regarding dependent claims 8 and 19, Mukhopadhyay discloses using a camera to record the 
presentation on page 2. 

Regarding dependent claims 12 and 28, Mukhopadhyay discloses comparing color histograms 
of successive frames with the slides on page 5, right column - page 6, left column. 
Regarding dependent claim 16, Mitchell discloses document images contain hyperlinks on col 
8, lines 60-67. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding independent claims 17 and 21, Mukhopadhyay discloses "link between a document 
file and a segment of a video file responsive to the comparison of a transformed document and 
transformed video frame", on page 5, see figure 7 teaches matching slides in each frame to 
synchronize video data to the slides). 

However, Mukhopadhyay does not explicitly disclose "a scanned document". 

Mitchell discloses "a scanned document" on col. 8, lines 36-43 and col. 9, lines 18-19 
teaches document pages are scanned into images. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
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Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 18, Mitchell discloses "a scanner to create scanned document from 
a physical document", on col. 8, lines 36-43 and col. 9, lines 18-19 teaches document pages are 
scanned into images using OCR. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 20, Mukhopadhyay discloses a Lecture Browser user interface for 
viewing slides and video on page 2, see figure 2. 

Regarding dependent claim 24, Mitchell discloses removing borders from the scanned 
document on col 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 
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5. Claims 5-7, 9-11, and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Mukhopadhyay and Mitchell, as applied to claims 1 and 21 above, further in view of 
Kumar (USPN 5,835,129 - issued on 11/1998), 

Regarding dependent claims 5-6, 9-10, and 22, Mukhopadhyay and Mitchell disclose the 
invention substantially as claimed as described supra. However, Mukhopadhyay and Mitchell 
do not explicitly disclose "orthonormal transform" and "discrete cosine transform". 

Kumar discloses transformation of the image frame and the process of dividing image 
frames into blocks using DCT (discrete cosine transform) defined by the orthonormal basis on 
col 8, line 60 - col. 9, line 3. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Kumar into Mukhopadhyay and Mitchell to provide the 
transformation of an image using DCT defined by the orthonormal basis, as taught by Kumar, 
incorporated into the images of Mukhopadhyay and Mitchell, in order to facilitate the division of 
image frames into blocks. 

Regarding dependent claims 7 and 11, Mitchell discloses removing borders from the scanned 
document on col 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay and image of Kumar, in order to provide a low-cost and high performance 
solution for converting paper into a form that can be accessed through the network such as the 
Internet or a media such a disk or CD-ROM. 
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6. Claims 14-15 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mukhopadhyay and Mitchell, as applied to claims 1 and 21 above, further in view of 
Bloomberg (USPN 5,181,255 - issued on 01/1993). 

Regarding dependent claims 14-15 and 26, Mukhopadhyay and Mitchell disclose the invention 
substantially as claimed as described supra. However, Mukhopadhyay and Mitchell do not 
explicitly disclose "extracting handwritten annotations from scanned documents". 

Bloomberg discloses extracting handwritten annotations from a scanned image on col. 1, 
lines 24-33, see Abstract. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Bloomberg into Mukhopadhyay and Mitchell to provide a 
way to extract handwritten annotations from a scanned image, as taught by Bloomberg, 
incorporated into the images of Mukhopadhyay and Mitchell, in order to provide an improved 
system to identify and separate printed text and handwritten annotations from an image. 
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Response to Arguments 

7. Applicant's arguments filed 6/03/04 have been fully considered but they are not 
persuasive. 

Regarding Applicant's remarks on pages 6-7: 

Applicant argues that Mukhopadhyay does not teach "linking a document to a video", 
however, Mukhopadhyay on page 2, right column, page 3, left column, and page 5, see Figure 7: 
teaches slides (document) are matched and synchronized with the video, in other words, the 
slides can be "connected" and related to the video, frame- by-frame. 

Regarding Applicant's remarks on pages 7-8: 

Applicant argues that Mukhopadhyay does not teach "transforming the scanned 
document into a scanned document identifier". 

Mukhopadhyay discloses each slide image is set with a value or identifier to facilitate the 
matching of each slide to each video frame (on page 5, see figure 7). Even though, 
Mukhopadhyay does not disclose the "transformation of the scanned document", Mitchell 
discloses the translation of paper to an electronic document (col. 6, lines 36-40). Furthermore, 
Mitchell discloses when the electronic document such as HTML document is created from 
scanning the paper document, an HTML address (identifier) is used to view the representative 
document of the scanned HTML document (see col. 7, lines 7-14). 

Applicant argues that Mukhopadhyay does not teach "obtaining a video file having a 
plurality of video frames", however, Mukhopadhyay does disclose "a video file having video 
frames" on page 5 teaches ". . .every frame of the overview video and matching each frame 
against all slide images" and Figure 7 shows a video sequence of a plurality of segments to 
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facilitate the matching to slides. Furthermore, Mukhopadhyay on page 6 teaches methods for 
computing which slide corresponds to the video frame (See Figure 8). 
Regarding Applicant's remarks on page 9: 

Applicant argues that Mukhopadhyay, Mitchell, and Kumar do not teach "orthonormal 
transforms which are performed at once on the entire scanned document". 

Kumar discloses the orthonormal transformation on an image frame on col. 8, line 60- 
col. 9, line 3, however, Kumar's invention can also transform still images, electronic white 
boards, slides (see col 4, lines 30-38), in other words, Kumar is not limited to only image 
frames. 

Applicant argues that Mukhopadhyay, Mitchell, and Kumar do not teach "removing the 
least significant information", however, Mitchell on col. 8, lines 36-43 does teach removing or 
cropping information such as borders from the scanned document. The claimed limitation can be 
interpreted as "removing any type of information from the scanned image that is not necessary to 
be able to enhance the transmission of images over the network (Mitchell col. 8, lines 43-47 
motivation for removal). 

Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
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the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 



CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 



examiner should be directed to Almari Yuan whose telephone number is 703-305-5945 (571- 
272-4104 after October 20, 2004). The examiner can normally be reached on Mondays - Fridays 
(8:30am -5:00pm). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Feild, can be reached on 703-305-9792 (571-272-4090 after October 20, 
2004). The fax phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-2 1 7-9 1 97 (toll-free). 



will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 



Any inquiry concerning this communication or earlier communications from the 



Joseph feild 
supervisory patent examiner 
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